Predictive information, multi-information, and binding information

نویسندگان

  • Samer Abdallah
  • Mark Plumbley
چکیده

We introduce an information theoretic measure of dependency between multiple random variables, called ‘binding information’ and compare it with several previously proposed measures of statistical complexity, including excess entropy, Crutchfield et al’s entropy of causal states, Bialek et al’s predictive information, Dubnov’s information rate, and the multiinformation. We discuss and prove some of the properties of the binding information, particularly in relation to the multi-information, and show that, in the case of sets of binary random variables, the processes which maximises binding information are the ‘parity’ processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity. Predictive information, multi-information, and binding information Samer Abdallah ([email protected]) Mark Plumbley ([email protected]) December 9, 2010 Abstract We introduce an information theoretic measure of dependency between multiple random variables, called ‘binding information’ and compare it with several previously proposed measures of statistical complexity, including excess entropy, Crutchfield et al’s entropy of causal states, Bialek et al’s predictive information, Dubnov’s information rate, and the multiinformation. We discuss and prove some of the properties of the binding information, particularly in relation to the multi-information, and show that, in the case of sets of binary random variables, the processes which maximises binding information are the ‘parity’ processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity.We introduce an information theoretic measure of dependency between multiple random variables, called ‘binding information’ and compare it with several previously proposed measures of statistical complexity, including excess entropy, Crutchfield et al’s entropy of causal states, Bialek et al’s predictive information, Dubnov’s information rate, and the multiinformation. We discuss and prove some of the properties of the binding information, particularly in relation to the multi-information, and show that, in the case of sets of binary random variables, the processes which maximises binding information are the ‘parity’ processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Feature Selection Using Multi Objective Genetic Algorithm with Support Vector Machine

Different approaches have been proposed for feature selection to obtain suitable features subset among all features. These methods search feature space for feature subsets which satisfies some criteria or optimizes several objective functions. The objective functions are divided into two main groups: filter and wrapper methods.  In filter methods, features subsets are selected due to some measu...

متن کامل

A measure of statistical complexity based on predictive information

We introduce an information theoretic measure of statistical structure, called ‘binding information’, for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.’s predictive information, and the multi-information. We derive some of the properties of the binding information, particularly in relation to the multi-information, and...

متن کامل

Applicability of BAT Model for Children Information Search Behavior in some Preschools in Tehran

Background and Aim: This study is set to represent information search process in the selected preschool children in Tehran (namely Mahgol and Taranom) and compare the results with BAT model. Methods: This is an Applied and comparative study with qualitative approach based on grounded theory. Research population was preschool children in Tehran from two different regions of city. Sampling was do...

متن کامل

3D Classification of Urban Features Based on Integration of Structural and Spectral Information from UAV Imagery

Three-dimensional classification of urban features is one of the important tools for urban management and the basis of many analyzes in photogrammetry and remote sensing. Therefore, it is applied in many applications such as planning, urban management and disaster management. In this study, dense point clouds extracted from dense image matching is applied for classification in urban areas. Appl...

متن کامل

Decoherence effects on quantum Fisher information of multi-qubit W states

Quantum fisher information of a parameter characterizing the sensitivity of a state with respect to parameter changes. In this paper, we study the quantum fisher information of the W state for four, five, six and seven particles in decoherence channels, such as amplitude damping, phase damping and depolarizing channel. Using Krauss operators for decoherence channels components, we investigate t...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2010